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The nature of science aspects (NOS) are the most emphasized theme in 
many curriculum and reform attempts. Teachers as a curriculum 
implementer in the classroom play important role for policy, arrangements 
and classroom experiences on NOS understandings of students. Researchers 
generally assessed the teachers’ NOS conceptions using open-ended 
questions plus interview in limited samples. However, in the literature, there 
have been some critics on this way of assessment. For that reason, this study 
aims to assess elementary science teachers’ (N=47) NOS conceptions by 
using “Open-ended questions” and “Knowledge test”. The results showed 
existence of some misunderstandings on NOS and the participants’ 
responses to knowledge test and open-ended questions gave similar pattern 
in terms often aspects of NOS, except for three aspects. 

Keywords: Elementary Science Teachers, Nature of Science, Assessment of 
NOS 

Introduction 

Scientifically literate society of future requires people to have informed decision making 
ability with the knowledge about science and its aspects. Today, informed decision making 
ability on daily life situations or problems has been emphasized to learn and use in many 
educational studies (Mbajiorgu & Ali, 2003; Damastes & Wandersee, 1992; Klymkowsky, 
Garvin-Doxas & Zeilik, 2003). Although there have been many studies in the literature 
focusing on the components of informed decision making process in different fields of study, 
“aspects of nature of science” as an issue of informed decision making emerged and took 
much interest as one of the most studied issue in science education research (Cooper, Cowie 
& Jones, 2010; Dekkars, 2005; Marra & Palmer, 2005; Lederman, 2007; Palmquist & Finley, 
1997; McComas, 1998). Due to the importance of nature of science for scientifically literate 
society, the nature of science similar to scientific literacy has also been determined as an 
important aim of science education in many curriculums and international examination 
frameworks (Turkish Ministry of Education, 2007; AAAS, 1994; Kjaernsli & Lie, 2004; 
Olsen, Kjaernsli & Lie, 2005; OECD, 2003). Nature of science includes many aspects from 
scientific method to science in society for science education. The results of epistemological 
and educational studies indicated that there are commonly accepted aspects to teach about 
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nature of science in formal education (McComas, 1998; Khishfe & Lederman, 2006). These 
nature of science aspects are presented in the followings: 

1. Scientific knowledge is theory-laden. 

2. Scientific knowledge is tentative. 

3. Observation is different from inference. 

4. Scientific knowledge is based on evidence and observation. 

5. There is no hierarchy among hypothesis, theory and law. 

6. Laws and theories have different roles in science. 

7. Scientific knowledge is embedded in social and cultural context. 

8. Science is a way of knowing. 

9. There is no universally accepted one way to do science. 

10. Creativeness and imagination are also important to produce scientific knowledge. 

11. Scientist is not objective when s/he begins to study; or when s/he has a background 
(McComas, 1998; Lederman, Abd-El-Khalick, Bell, & Schwartz, 2002). 

For a long time, many of science lessons, textbooks and subjects have been beginning with 
nature of science issues and continued with content knowledge after nature of science part. In 
spite of these priority and consideration, research has consistently showed that there are 
inadequate explanations about nature of science aspects in textbooks and naive 
understandings in the minds of students (Blanco & Niaz, 1997; Marra & Palmer, 2005; Tsai, 
2006; Irez, 2006; Ryan & Aikenhead, 1992; McComas, 2003). In addition, nature of science 
as a school subject is not understood well enough by students, teachers and teacher educators 
(Irez, 2006; Thye & Kwen, 2003; Tsai, 2006; McComas, 2003; Sandoval & Morrison, 2003; 
Dagher & Boujaoude, 2005; Blanco & Niaz, 1997; Dogan & Abd-El- Khalick, 2008). 
Especially, teachers, among these groups, have an important role for organizing and 
conducting explicit-reflective instruction on and providing important experiences to help the 
students learn NOS aspects. Previous studies showed that there is no convincing evidence or 
tendency for transferring of teachers’ understandings of NOS into their classroom practices 
and students’ NOS understandings (Eederman & Zeidler, 1987; Akerson, Abd-El-Khalick, 
Eederman, 2000). In spite of lack of evidence, considering the fact that teachers are the key 
persons for implementing the science curriculum in classrooms and they are still thought to be 
important person in development of students’ understandings of NOS aspects (as cited in 
Akerson et ah, 2000). Practically determination of teachers’ understandings on NOS aspects 
in large scale studies might provide data for teacher profiles. Then, these profiles can be used 
for decisions of authorities on the factors which are effective in integration of explicit- 
reflective teaching into classroom practices. When considered the current studies about nature 
of science with teacher or prospective teachers, it can be seen that many of the studies focused 
on determination or assessment of the NOS aspects by using open-ended questionnaire and 
interview. Some studies are also focused on examining the effects of interventions including 
implicit, explicit or embedding strategies on participants’ NOS views with the help of an 
open-ended questionnaire and interviews (Akerson et ah, 2000, Morrrison, Raab, & Ingram, 
2009). The most widely used questionnaires in many studies are VNOS versions developed 
by Eederman et al. (2002), and Abd-El-Khalick (1998). The administration of the 
questionnaire was not seen enough to get comprehensive data, so the need for a follow-up 
interview became inevitable (Eederman et ah, 2002; Eederman, Wade & Bell, 1998). When 
taken into consideration of questionnaire plus follow-up interviews, it is seen that they are 
time and effort consuming, require having ability and knowledge to use complex qualitative 
coding, analyzing and interpretation techniques. For instance; VNOS-C application takes 45- 
60 minutes and follow-up interview adds more time to the application (Eiang, Chen, Chen, 
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Kaya, Adams, Macklin & Ebenezer, 2008). In addition, working with more participants is a 
difficult task for any researcher if s/he uses both open-ended questionnaire and follow-up 
interview. What is more, the questions of VNOS versions include generally generic questions 
in spite of their comprehensive nature to obtain data. For that reason, researchers might 
obtain different data from the focus of the question. All of these limitations mentioned above 
were also stated by Wenning (2006) and Liang et al. (2008). In the literature, there are few 
examples using only standardized test to study relationship of NOS aspects with achievement, 
age, gender and academic background. For example. Wood (1972) studied relationship of 
NOS conceptions of 443 pre-service teachers with gender and academic background by using 
WISP (Wisconsin Inventory of Science Processes). The author found only little relationship 
between NOS conceptions and gender, number of university courses taken and year in high 
school science. Using same inventory Carey and Strauss (1969) explored the relationship 
between NOS conceptions of 17 pre-service teachers and academic variables including grades 
achieved in science and number of credits taken. The authors did not find any significant 
relationship. When they increased the sample of the study to 221 pre-service teachers, they 
found that WISP scores were related academic variables including college science grade 
mean, total college grade mean (Carey & Strauss, 1969). In another study, Billeh and Hassan 
(1975) developed a test including 60 multiple-choice items on four major NOS aspects. They 
found reliability of the test as .58 and used this test to examine the relationship of gain scores 
of teachers on NOS conceptions and other variables including subjects the participants taught, 
teaching experience and previous in-service teaching. The authors found no significant 
relationship between NOS gain scores and any investigated variables. The studies of Billeh 
and Hassan (1975), Wood (1972) and Carey and Strauss (1969) used only test to study NOS 
aspects and they presented conflicting results. So, limitations of using only the tests should be 
considered to get more complete picture of the situation. As seen, using only either test 
approach or open-ended questions plus interview even if open-ended questions were 
experimentally prepared has limitations for the inferential statistics and generalizability of the 
results. Similarly, using six, seven or eight open-ended questions with interview has problems 
about content validity and nature of data (categorical data). As the important point in the 
assessment of the NOS aspects, domain specificity should also be taken into consideration for 
determining naive understandings on the NOS aspects. NOS aspects are emphasized as 
cognitive variables in the literature (Khisfe & Lederman, 2006; Khisfe & Abd-El-Khalick, 
2002). To measure cognitive variables with specifications, multiple-choice format has many 
advantages (Haladayna, 1997). Multiple-choice tests have practical importance and can easily 
be constructed with specific objectives. The application and scoring procedure is easier than 
other types of assessment. If multiple choice tests are used with embedded open-ended 
questions by considering representative part of sample, it might decrease some problems such 
as use of generic items, problems for practical use. By this way, it is possible to provide more 
appropriate measure for cognitive variables. Using NOS knowledge test rather than interview 
and complementary open-ended questions with the test items on NOS might provide 
opportunities for practical purposes in assessment of teachers’ understandings of the aspects 
of NOS. Therefore, this study was conducted to determine elementary science teachers’ 
understandings of the NOS aspects by using NOS Knowledge Test and Open-ended 
Questions. So, the purpose of the present study was to investigate the following general 
research question: How are Turkish science teachers’ understandings about NOS aspects 
when knowledge test and open-ended questions were used complementarily? 
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Method 

Participants 

The study was conducted with 47 elementary science teachers worked in schools in Bolu and 
Ankara cities of Turkey. The participants were selected by using convenience sampling since 
this way provide opportunities for cost, time and energy consumption. Turkey does not have 
states and is governed from the center of the one state. Therefore, decisions about sharing 
resources, money and assignment of administration are made from Ankara. Ankara as a 
large-scale middle-region city is the capital city of Turkey and the schools of Ankara have 
more resources and educational opportunity which cannot be found in the region of Turkey. 
For instance; science and technology centers, universities more than four and Teaching 
Material Development Center of the Turkish Ministry of Education exist in Ankara. 
Therefore, teachers have many opportunities to improve their abilities and knowledge by only 
visiting there. However, Bolu is a middle-scale north-western city of the Turkey and has less 
educational opportunities than Ankara. But, Bolu has one university and many places to visit 
for nature and environmental education. The participants’ ages range from 25 to 56. Twenty 
of the participants were male and 27 were female. Thirteen of them graduated from 
elementary science education departments of education faculties while 11 science teachers 
were enrolled in secondary science education departments of education faculties. Apart from 
these, 13 participants graduated from biology, chemistry and physics departments of science 
and art faculties. The participants (N=5) who are older than others graduated from education 
institutes while one participant graduated from chemistry engineering. Over the half of the 
participants had experience over 10 years (N= 24), while 23 participants’ experience were 
below the 10 years. The participants were asked if they join in any activities about history, 
philosophy, epistemology of science before. Twenty-eight participants stated no participation 
in these activities while the others joined at least one activity. 

Instruments 

In this study, quantitative research approach supported by qualitative data was used. To 
collect data from participants, NOS knowledge test, and open-ended questions were used. For 
the development of NOS knowledge test, firstly 60 items were constructed based on the 
related literature (McComas, 1998; Lederman et ah, 2002; Kihsfe & Lederman, 2006; Khsife 
& Abd-El-Khalick, 2002). Table 1 displays each NOS aspect and corresponding item 
numbers in the NOS knowledge test. 

Table 1. NOS aspect and corresponding item numbers in the NOS knowledge test 


Aspects of Nature of Science 

Scientific knowledge is tentative 

Scientific knowledge is based on evidence and observation 

Observation is different from interference 

There is no hierarchy among hypothesis, theory and law 

Scientific knowledge is theory-laden 

Scientific knowledge is embedded in social and cultural 

context. 

There is no universally accepted one way to do science 
Laws and theories have different roles in science 
Creativeness and imagination are also important to produce 
scientific knowledge 


Items 

8,9, 10, 11, 12, 13 
14,15,16,17,18,19,20,21 
22, 23, 24, 25, 26 
27,28,29, 30,31 
32, 33, 34, 35,36,37 
38, 39, 40,41,42 

1,2, 3, 4, 5, 6, 7 
43, 44, 45, 46 
47, 48, 49 
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Scientist is not objective when he or she begins to study, he or 
she has a background 
Science is a way of knowing 


50,51,52, 53 

54, 55, 56, 57, 58, 59, 
60 


The questions were structured as three-choice items with '"True”, "False” and "There are both 
true and false parts”. In addition to the three-choice items, the participants were asked to 
explain their reasons about the answers given to the items numbered as 7, 13, 21, 26, 31, 37, 
42, 46, 49, 53, and 60. Therefore, a total of 11 open-ended questions for reasons were also 
added in the test to obtain more focused data related to participants’ understanding of the 
NOS aspects. Then, the open-ended answers were analyzed; representative number of the 
participants’ writings were selected for practical purposes (N=22, 46%). The participants 
were asked to explain their reasons to the answer given for the following items: 

a) Scientific knowledge is tentative. 

b) Observation is different from inference. 

c) Scientific knowledge is based on evidence and observation. 

d) There is no hierarchy among hypothesis, theory and law. 

e) Laws and theories have different roles in science. 

f) Scientific knowledge is embedded in social and cultural context. 

g) Science is a way of knowing. 

h) There is no universally accepted one way to do science. 

i) Creativeness and imagination are also important to produce scientific knowledge. 

j) Scientist is not objective when he or she begins to study; he or she has a background. 

For the content validity of the test, the opinion of one expert in the field of science education 
was considered. The three-choice part of the test was analyzed by using ITEMAN for the item 
and test characteristics. The items of 7, 13, 21, 26, 31, 37, 42, 46, 49, 53, and 60 were not 
considered for the analysis because they were focus questions for the NOS aspects related to 
them and covered all items of the NOS aspects within the same category. Some of the items 
were eliminated due to their discrimination index (Point-biserial correlation coefficients) 
values below .20 (Kehoe, 1995). Then, mean P value as an index for the test difficulty and 
alpha value were investigated. After the elimination of the items, the final version consisted of 
29 items. The selected item numbers can be seen in table 2. The minimum score is 2 while 
maximum score is 56 in the scoring way of two-point for each correct answer. The mean P 
value as difficulty index was found to be .40 for the test while alpha value for whole test was 
.79. Mean biserial correlation value for whole test was .66. The item analysis showed that 
there is no item corresponding for the aspect of theory-laden science. Therefore, this aspect 
was also eliminated from the open-ended questions. Therefore, the final form of the 
instrument included 29 three-choice and 10 open-ended items. All of the items are at the level 
of knowledge category for taxonomy of cognitive abilities. Final version of the three-choice 
item examples and open-ended item example can be seen in the tables 3 and 4. 
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Table 2. Table of specifications for the nature of science knowledge test (Final Form) 


Aspects of Nature of Science 

Scientific knowledge is tentative 

Scientific knowledge is based on evidence and observation 

Observation is different from interference 

There is no hierarchy among hypothesis, theory and law 

Scientific knowledge is embedded in social and cultural context. 

There is no universally accepted one way to do science 

Laws and theories have different roles in science 

Creativeness and imagination are also important to produce scientific 

knowledge 

Scientist is not objective when he or she begins to study, he or she 
has a background 

Science is a way of knowing 


Questions 

8,9,10,11,12, 13 
15,16,18,19, 21 
24, 26 

27,28,29,30, 31 
39, 40,41, 42 
1,2,3,5,6,7 
43, 46 
47,48, 49 

50,51,52, 53 

54, 60 


*Bold items were used for open-ended questions, so they were not used in the item analysis 
process. 


Table 3. The three-choice item examples of the final version of the nature of science 

knowledge test 

Items True False There are both true 

and false parts 

Observation and inference have the same meaning 

Theories might change over time 


Table 4. One open-ended item example of the final version of the nature of science 
knowledge test 


Items True False There are both true 

and false parts 

Scientific knowledge is tentative 

Could you explain reason for your choice about the question above? 


Analysis of the data 

To analyze data, total scores for each aspect in NOS knowledge test were used to categorize 
the participants by using the rule of original scale 0 for false answer, 2 for true one and 1 for 
the category of answer between them. After calculating the scores of the participants, means 
were used to categorize the participant group. The criteria for categorization by considering 
two decimals are established as 0-.44 for naive category, .45-1.44 for transitional, 1.45-2 for 
the informed ones. Such a categorization was used to provide original scale intervals. Original 
limits in the test have whole numbers for each category as 0, 1, and 2, so, decimals which 
completed to whole numbers were used as cut-off points in categorization. The relationship 
between original scale and categorization scale is illustrated in Figure 1. Therefore, original 
categorization in the test was also represented in the means for the group of the participants. 
By providing larger interval for “transition” category, a strict rule for categorization on naive 
and informed participants was also established to get a clear picture. Additionally, transitional 
category was re-categorized as approaching naive or approaching informed to provide more 


202 






detailed analysis and overcome vague results in this interval. To categorization of transitional 
understandings, the interval of .45-. 94 for “approaching naive” and the interval of .95-1.44 for 
“approaching informed” were used to show more detailed picture of the understandings. The 
figure 1 explains the categorization approach. To analyze the open-ended answers, 
categorization of the participants’ answers were conducted by using content analysis with the 
evaluation frame of Lederman et al. (2002). After the analysis of open-ended questions, 
categorization and mean score calculation by giving a score to each participant in line with 
their category and original scale and adding all scores, then, dividing the total score by 22 
were conducted for each aspect. Comparison of mean scores on test items and open ended 
questions is the basis for this study (see table 5 and table 6). 


Oiigitid Scdfi : 

□=NaiYie 

1= Trprsitionisl 

2^ Informed 

Corresp anding Cate gomatiorL Scale : 

T 

0-.44^0 

T 

.45-1.44^1 

T 

1 .45-2=2 


Rs-ci1^gcarm1ioarL of TtmsilifHiil CaLegciiy : .45- . 94— J^proiching riam .95-1.44— i^iosichmg iifciimsd. 

Figure 1. Relationship between original scale and corresponding categorization scale 

Results 

The results of the study will be presented in this section. The participants mean scores on each 
aspect can be seen in table 5. 

Table 5. The mean scores of corresponding categories of the participants in the knowledge 
test 


NOS Aspect 

Mean 

Score 

Category 

Transitional Participants’ 
Categorization 

Tentativeness 

1.4 

Transitional 

Approaching informed 

Evidence and Observation Based Science 

.65 

Transitional 

Approaching naive 

Observation and Inference Difference 

1.4 

Transitional 

Approaching informed 

No hierarchy among hypothesis, theory and law 

.1 

Naive 

- 

Social and cultural embeddedness 

1.14 

Transitional 

Approaching informed 

No universally accepted one way to do science 

.60 

Transitional 

Approaching naive 

Roles of Laws and Theories 

.87 

Transitional 

Approaching naive 

Creativeness and imagination 

1.5 

Informed 

- 

Subjectivity 

.77 

Transitional 

Approaching naive 

Definition of Science 

1.2 

Transitional 

Approaching informed 


Table 5 shows that Turkish elementary science teachers have naive ideas about the aspect of 
“hierarchy among hypothesis, theory and law” whereas they have informed ideas about the 
aspect of “creativeness and imagination” in science. In general, majority of the participants 
are at the transitional category for the other aspects of NOS with changing degrees. Detailed 
examination of the transitional category showed that the participants approached to informed 
category for the aspects of “tentativeness”, “observation and inference”, “social and cultural 
embeddedness” and “definition of science” whereas they approached toward naive category 
for the aspects of “ evidence and observation based science”, “no universally accepted one 
way to do science”, “roles of laws and theories” and “subjectivity”. 
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Table 6. The results of content analysis of the participants’ answers to the open-ended questions 
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Note;P; Participants, N: Naive, T: Transitional, 1: Informed 
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Table 6 presents categorization of the participants for the NOS aspects in terms of 
three categories as “Naive”, “Transitional” and “Informed”. The example definitions 
and explanations for each NOS aspect from their writings include the following 
excerpts: 

“Science is a study to reach correct knowledge ” 

“Observation is data gathered to see results of an experiment; inference is a result to 
be seen ” 

“Hypotheses and theories are established for explaining laws” 

“To reach scientific knowledge, there is a need to present observations, experiments 
and scientific evidence ” 

“Law and theory have the same meaning, but law is the more proven type ” 

"Scientific knowledge do not develop in social and cultural structure, they are 
produced by using scientific method and techniques ” 

“Scientific method is only one way to reach correct knowledge ” 

“Scientists cannot find correct answer without being objective” 

“Scientists and science use creativity and imagination at the beginning of any study” 
“Laws cannot be changed due to the fact that they are absolute” 

“If observations and data change, scientific knowledge also changes” 

When table 5 and table 6 are taken into account together, it can be seen that “No 
hierarchy between theory and law” aspect is a problematic issue for the teachers as 
indicated in the results of both open-ended question analysis and knowledge test 
answers. In terms of “tentativeness”, “empirically based science”, “observation and 
inference”, “social and cultural embeddedness”, “subjectivity” and “definition of 
science” aspects, data from open-ended questions supported the knowledge test result 
showing the “transitional” category for the participants. However, different results 
were also obtained from the analysis of open-ended questions and knowledge test 
items. For example, analysis of participants’ answers to the open-ended questions 
showed that majority of the teachers hold naive views regarding “no universally 
accepted one way to do science” (73%) and “roles of theories and laws” (68%) 
aspects whereas they were in transitional position regarding “creativeness and 
imagination”(41%) aspects of NOS. In spite of these differences on the transitional 
categories, re-examination of transitional categories in test score analysis provided 
more detailed picture by showing that the categories of the participants on the aspects 
of “no universally accepted one way to do science”, “roles of theories and laws” and 
“subjectivity” were determined as “approaching naive”. Analysis of participants’ 
open-ended answers showed that the participants hold naive views in terms of these 
NOS aspects. Categorizing the participants as “approaching naive” and “naive” on the 
aspects of NOS might be an evidence for complementary nature of the assessment 
including open-ended and multiple-choice knowledge test formats. Although they 
present some differences, two data sets give partially complementary results in terms 
of NOS aspects. Table 7 summarizes the scores on the each of the assessment ways to 
compare them easily. 
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Table 7. Mean scores on corresponding aspects for the “knowledge test” and “open- 
ended questions” 


NOS Aspects 

Assessment Approach 
Knowledge Test Open-ended 

Questions 

Complementary 

Aspects 

Tentativeness 

1.4 

.63 

© 

Evidence and observation based 

.65 

1 

0 

science 

Observation and inference difference 

1.4 

1.09 

0 

No hierarchy among hypothesis, 

.1 

.14 

© 

theory and law 

Social and cultural embeddedness 

1.14 

.68 

© 

No universally accepted one way to 

.60 

.32 

- 

do science 

Roles of laws and theories 

.87 

.18 


Creativeness and imagination 

1.5 

1.05 

- 

Subjectivity 

.77 

.48 

© 

Definition of science 

1.2 

.68 

© 


Conclusion 

The results of the study showed that science teachers had many naive understandings 
about the aspects of NOS. They had the most extreme naive understandings regarding 
relationship between theory and law. Most of the science teachers believed that there 
is hierarchy among hypothesis, theory and law. One of the possible reasons for this 
understanding might be explanations of theory, law and hypothesis as hierarchical and 
linear structures in the biology textbooks because NOS aspects are explained as a 
separate unit only in the biology textbooks (Irez, 2009). When the individual answers 
to open-ended questions were examined, it was seen that some of the teachers for 
each aspect presented at least more than one naive understanding. This result is 
consistent with previous studies (e.g. Akerson & Hanuscin, 2007; Akerson et ah, 
2000). For example, with a sample of three K-6 teachers Akerson and Hanuscin 
(2007) indicated that all teachers believed that theories become laws and laws are 
more certain than theories. In addition, teachers held the view that laws are proven 
whereas theories are unproven. Similar to present study, all participants claimed that 
science involves creativity and imagination. A similar result was reported in Morrison 
et al.’s (2009) study. They stated that most of the teachers had inadequate ideas 
regarding relationship between theories and laws while half of the teachers provided 
adequate ideas on the aspect of “creativity and imagination”. In addition, the 
participants presented intermediary ideas on the majority of other aspects of NOS. In 
another study, Akerson et al. (2000) examined the elementary teachers’ NOS views 
and found that majority of them had inadequate ideas on “theories and laws”. For the 
other aspects, the elementary teachers showed various categories from naive to 
transitional to informed without any certain tendency among the participants. The 
naive understandings of science teachers might be eliminated by reflection based in- 
service programs on teaching practices about the NOS aspects. In the literature, 
explicit-reflective teaching was shown to be effective for elimination of naive 
understandings (Khisfe & Lederman, 2006). In the science teacher education 
programs, more explicit activities with their rationales might also be effective to help 
science teachers question their naive understandings about the NOS aspects. 
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Therefore, explicit attention to the NOS aspects might be strengthened in pre-service 
education level by this way. 

As the important side of this study, the answers to knowledge test and open-ended 
questions gave partially complementary pattern in terms of ten aspects of NOS, 
except for the three aspects including “no universally accepted one way to do 
science”, “roles of theories and laws” and “creative and imaginativeness” aspects. Lin 
and Chen (2002) also stated that multiple-choice format is more conservative and is 
not enough to use only one tool to study NOS. Limitations of only using tests for 
NOS studies also emerged in the literature by showing conflicting results related to 
relation of NOS conceptions with content knowledge and other academical variables 
(Billeh & Hassan, 1975; Carey & Strauss, 1969; Wood, 1972). Both practical 
purposes and conservative nature of multiple-choice format make it as a valuable 
complementary tool to use with open-ended question format which has been used 
more frequently in the NOS studies. In fact, large interval for transitional position in 
original scoring presents a problem for test use, but the use of open-ended questions 
presents additional evidence of the transitional position in the interval determined. 
Re-categorization of transitional category is useful to both compare the results of the 
test and open-ended questions in detail. In addition, the cut-off limits for the intervals 
included sensitive decisions about the scores, for example; one aspect might be in 
limits of the interval, in such situations, use of open-ended questions might provide 
complementary and supportive data to make decision. In general, two data sets are 
complementary to each other. This result might provide an evidence to use this way of 
assessing the NOS aspects for practical purposes with more participants in large scale 
studies. In the future studies, supported NOS aspects by both of the data types- 
quantitative and qualitative- can be used for generalizing the results for larger 
populations. The approach also provided to see tendencies of the group for each 
aspect and individual understandings of representative part of the group on each 
aspect. 

Limitations of the study 

This study was subject to some limitations. The results of this study should be 
examined with care due to limitations for sample of the present study and the 
instruments used. Further research on validation of the instrument needs to be 
continued. By using the same approach, knowledge test combined with open-ended 
questions on NOS should be applied to large sample size and the results should also 
be compared with the data collected by frequently used instruments such as VNOS-C 
and VOSTS to examine validity of the results. Usability of this way of assessment on 
NOS aspects by the teachers should also be investigated. 
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